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ABSTPACT 



This study investigated the kinds of knowledge and 
strategies beginning second- language readers used when reading texts 
to build their own consistent patterns of meaning and experience. 
Specifically, it looked at how readers* knowledge grew, where their 
ideas originated, and when and how their ideas were used. Subjects 
were five college students of Arabic, who were instructed in the 
think-aloud procedure and who then read four informative texts with 
varying text structures, styles, and subjects, thinking aloud as they 
read. Subject comments were categorized by reasoning operation and by 
monitoring concerns occurring during reading. Three additional 
analyses examined strategies used in meaning development, knowledge 
source used in making meaning, and timing of the comment in relation 
to the reading experience. Results indicate that subjects' primary 
concern during reading was with meanings they were developing, by 
using hypothesizing and schemata, formulating and refining ideas, 
focusing on specific content, and validating the text-model or 
text-world being developed. Behavior patterns differed somewhat at 
different points in the reading process, with more questioning and 
hypothesizing at the beginning and more knowledge use later. 
Behaviors generally focused on awareness rather than use. 
Meaning-construction strategies were consistent across texts. 
Contains 28 references. (MSE) 
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Verbal Protocol Data As A Reliable Source Of Information 
About Reading Behavior And Cognitive Processes: The Case 
Of Reading Arabic As A Foreign Language 

Introduction 

Reading is a cognitive activity, a process of meaning- 
in~motion in which understandings are developed and changed 
throughout the process, reflecting an interaction between the 
textual and extratextual factor at work in the process. 

When readers develop their ideas, they rely on various 
kinds of knowledge: knowledge about the content, genre, and 
structure and how these work together in the evolution of the 
conceptualization of an entire unit of discourse (Iser, 1975; 
Langer, 1986; Langer, et al . , 1990), This approach to the uss 
of knowledge grows out of the constructivist view that 
language comprehension involves active participation on the 
part of the reader (Bartlett, 1932; Bransford & Johnson, 
1973; Spiro, 1980) and that the meaning that develops is the 
consequence of a wide range of textual, contextual, and 
attitudinal forces continually at play in the human mind. 
This view of language comprehension has been adopted by many 
contemporary linguists (Fillmore, 1981; Halliday, 1977; van 
Dijk, 1977) and psychologists (Minsky, 1975; Shank & Abelson, 
1977; Rumelhart & Ortony, 1977; Spiro, 1980). 
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This approach to the use of knowledge is consonant with 
the proposition that meaning construction in reading is a 
function of the interrelationships among a variety of complex 
sources, and leads to the contention that "the act of meaning 
construction cannot be described by a linear, or even a 
simply recursive model-~interpretation and change are deeply 
interwoven every step of the way" (Langer, 1986:74). 

With respect to the construct of reading comprehension, 
it is known that direct assessment of the trait is impossible 
since it is a mental operation which is unobservable . Using 
think-aloud protocols is a way of getting at the unobservable 
behavior of reading comprehension. 

Verbal Reports 

The concept of introspection, that is, the manner in 
which the mind observes its own processes, can be traced at 
least to Aristotale and Plato (Boring, 1953) . However, 
concurrent verbal reports, a popular tool of reading 
researchers, had their genesis in the experimental intro- 
spection of classical psychology. 

The literature of cognitive psychology reports 
significant differences in opinion regarding the validity and 
reliability of verbal reprots. A think-aloud protocol is 
produced when a reader verbalizes his or her thought 
processes while completing a given task. 
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In a widely cited critique of the method, Nisbett and 
Wilson (1977) content that all verbal report data are highly 
suspect since, according to them, people have little or no 
access to their higher order cognitive processes. 

Others believed that verbal reports might be appropriate 
under certain conditions . These researchers maintained that 
such data were useful in exploratory studies and as means of 
identifying hypotheses which could then be examined in 
subsequent research (Kanfer, 1968) , or as one of several data 
bases providing convergent validity for a given research 
questions (Aweiss, 1993b; Sternberg, 1985) . However, they 
cautioned against viewing verbal reports as substitutes for 
the events which they describe. 

Still other researcher in the field (Afflerbach and 
Johnston, 1984; among others) share a perspective most 
cogently expressed by Ericsson and Simon who said that 
"verbal reports, elicited with care and interpreted with full 
understanding of the circumstances under which they were 
obtained, are a valuable and thoroughly reliable source of 
information about cognitive processes" (180:247). 
Specifically, Ericsson and Simon (1980) stressed the 
importance of minimizing: (a) the additional cognitive 
demands imposed by the directions to verbalize and (b) the 
amount of time between the utilization of a strategy and its 
report. When the two guidelines are implemented, it is less 
likely that the act of verbalizing will substantially alter 

o 5 
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the subject's normal thought processes and more likely that 
the verbal reports will accurately reflect those processes. 
In short, the validity of the verbalizations will be 
enhanced. 

Verbal reports or think-aloud protocols have been used 
in many LI and L2 research studies as a method of getting at 
the mental processes that language learners use (Afflerbach & 
Johnston, 1985; Bereiter & Bird, 1985; Block, 1986; Cohen, 
1987; Cohen & Hosenfeld, 1981; Ericson & Simon, 1984; Langer, 
1986; Sarig, 1987; among others) . 

Despite having some reservation about the think aloud 
procedure--would slow down the reading process and provide an 
incomplete picture of strategy actitivty--Bereiter and Bird 
(1985) concluded that it is a useful technique, especially 
when the focus is the identification of strategies to teach. 
Likewise, after using the procedure in a reading experiment, 
Johnston and Afflerbach (1985:229) concluded that the think 
aloud procedure "elicited. largely veridical indications of 
the process " . 

In summary, studies that utilized introspective 
approaches were desinged to gain insight into various aspects 
of the reading process. While the results of these studies 
are not conclusive, they do suggest that under certain 
conditions, introspective approaches can be a viable means o.: 
studying the reading process. 
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The Study 

The research investigated in this paper is part of a 
larger investigation that looked into the reading behavior 
and cognitive processes of foregin language learners. 
Subjects: The participants for this study were beginning 
American learners of Arabic as a foreign language (AFL) 
enrolled at a large midwestern state university. 
Materials: Materials for the study consisted of four short 
texts (150-175) informative texts adapted from the EMSA 
textbook (Peter Abboud & Ernest McCarus, Eds.). The passages 
are written in a variety of styles (text structures) about 
different subjects. 

Procedure: The five subjects (2 males and 3 females) who 
voluntraily took part in the think aloud procedure were 
individually briefed and trained. The training session lasted 
between 13-30 minutes and included a sequence of direct 
explanations, coahing, and modeling. The subjects were 
instructed to "say out loud everything that comes to mind as 
you try to read and comprehend the text . " The experimenter 
first provided an example and modeled thinking aloud. The 
subjects were then individually asked to think aloud about a 
short passage, and the experimenter provided feedback and in 
some cases more modeling. Subjects practiced thinking aloud 
on two more short passages. No additional practice was 
needed. During the actual experiment. The readers were not 
cued at predetermined points. Once they had received their 
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initial instruction encouraging them to express the processes 
tehy use while reading, they were not prompted to verbalize 
their thinking processes . 

Analysis of Meaning Construction 
The purpose of this analysis was to explore the kinds of 
knowledge and strategies readers used when contemplating 
written texts to build their own consistent patterns of 
meaning and experience. In particular, this analysis 
investigated how foreign language readers' knowledge grew, 
where their ideas originated, and when and how their ideas 
were used. 

The on-line protocols gathered for the present study, 
provide the most direct look at such processes. By their very 
nature, such analyses enable the researcher to look through 
what readers say in order to describe the underlying patterns 
of foreign language readers' behavior. 

To permit the codification and analysis of subjects' on- 
line attempts to construct meaning when they read, a detailed 
system, adapted from Langer's (1986) Analysis of Meaning 
Construction was used. The system describes the knowledge 
sources, specific strategies, and general approaches readers 
use to develop their interpretation of the text. Each comment 
was categorized along two major dimensions : on the reasoning 
operations, and on the monitoring concerns that occur during 
the reading activity. An additional three analyses examined 
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(1) the strategies used in meaning development, (2) the 
knowledge source used in making meaning, and (3) the time in 
the reading experience when that particular comment occurred. 

The major categories examined in the Analysis of Meaning 
Construction ar^ summarized in Figure 4) . 



Figure 4 

Analysis Of Meaning Construction 



Reasoning Operations 

Questioning 

Hypothesizing 

Assuming 

Using Schemata 

Making Metacomments 

Citing Evidence 

Validating 



Strategies 

Generating Ideas 
Formulating Ideas 
Revising 
Evaluating 



Monitoring 

(awareness and use) 
Task goals 
Task Subgoals 
Genre/Discourse Structure 
Mechanics 
Lexicon 

Statements of Meaning 
Refinements of Meaning 

Time 

Before 
During 
After 



Knowledge Source 

Genre 

Content 

Text 
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The analyses were concerned with how readers make 
meaning, as this vras revealed in their think-aloud comments 
about their behavior while reading. 

Patterns of Reasoning Operations 
One aspect of readers' strategic behavior investigated 
in this study, is the pattern of reasoning that is involved 
in meaning construction. During the process of independent 
reading, readers ask questions, make assumptions about what 
they and others know, use information or ideas drawn from 
their general schematic knowledge, generate hypotheses, give 
evidence for and seek validation of their ideas. The 
following is a sample of the thoughtful reasoning behaviors 
involved in the construction of meaning in the four 
experimental texts used in this study. The categories as well 
as the explanation that follow each of the reasoning 
operations were adapted from the original system of meaning 
construction developed by Langer (1986) . The examples cited 
reflect comments made by the subjects in this study (Figure 
5). Readers' communication units-~remarks and comments-- (193 
in all) were transcribed and tabulated (by the researcher and 
an assistant) into 13 major categories and subcategories (see 
Figure 5) . 



10 



Verbal Protocol Data 



9 



Figure 5 
Reasoning Operations in Reading 



Questionlnsf- uncertainties and incomplete ideas that the 
reader has at any point in reading- -related to the genre, 
content, or text (no specified guess or expectation) . "why is 
he telling the president about his teacher?" "What's it 
saying about the summer?" 

Hypothesizing 1 Present- plans or expectations the reader 
makes about what information will be presented at that point 
in time. "So, he is learning how to teach and he is writing to 
the president of U of Kuwait," "Perhaps this means taught, so 
he must have taught the Middle Eastern stuff." 
(H) Hypothesizing 2 Future- plans the reader makes about 
what will be "said" in succeeding portions of the text. "I 
think he will go back to or travel to Jerusalem. " 
(H) Hypothesizing 3 Past - plans the reader makes about 
what might now be "said" or "understood" in preceding 
portions of the text. "May be it meant something like "speak, 
I speak . " 

(A) Assuming- meanings the reader assumes need no further 
explanations or elaborations. "Right, I thought it was about 
Egypt," "It doesn't seem to be an important thing, " "There is 
nothing to think about." 
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(S) UBlng Schemata- the ideas being developed or explained 
based on genre, content, or text. "Oh, that's in upper 
Egypt," "that's similar to what we have in the states." 
(SP) Using Schexaata, Personalized- personal experience 
drawn upon by the reader. "Ya, I've seen it, it's large," "I 
know that famous saying historically. " 

(SE) Using Schemata Evaluated- evaluations and judgments 
being made about what is being read. "It's the capital of 
Egypt, I know that," "He's visiting Jerusalem, that's 
pleasant . " 

(SL) Making Schematic Links- concept links being made by 
the reader. "He met me in the rain, that's strange. I thought 
it doesn't rain there in summer," "didn't he say he was 
Egyptian. " 

(MC) Making Metacomments Content- comments about the 
reader's use or non-use of particular content information, 
"that's not a clear letter," "Ah, there it is, I didn't read 
the whole sentence." 

(MT) Making Metacomments Text- comment about the reader's 
use or non-use of particular surface features of the text 
itself. "This is difficult, I can't read it," "the English 
word is more difficult than the Arabic word." 
(E) Citing Evidence- the information the reader presents, 
the explanations provided, or the evidence developed to 
answer a question or carry out a hypothesis, "he's looking 
for a job," "such an industrious student, this guy." 
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Validatirxg- information ( implied or direct) that the plan 
was fulfilled or a decision made. "Oh, Herodotus, that's the 
Greek guy," "T see, that's how it works there." 



General Patterns 
When foreign language learners read, the focus of their 
attention is almost equally divided between hypothesizing and 
using schemata, that together, make up more than half of 
their reasoning operations (Table 28) . Other mental 
operations take place but in lesser frequency. Across the 
reading texts in the present study, 56 (29%) of readers' 
remarks (communication units) involved the direct use of 
hypothesizing — present hypothesizing accounting for 87% of 
their remarks within that category. 45 (23.6%) of subjects' 
comments involved using schemata. The remaining comments were 
split primarily among making metacomments 35 (18%), 
questioning 29 (15.2%). The reading protocols also showed 
less concern citing evidence in support of interpretations 
being formulated 6 (2.8%) and more with validating previous 
interpretations 22 (11.4 %) . 
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Table 28 
Reasoning Operations 



Number & % o 


f Remarks /Comments (Communication Units) 


Hypothesizing 


56 


(29%) 


Using Schemata 


45 


(23 .6%) 


Metacomments 


35 


(18%) 


Questioning 


29 


(15.2%) 


Citing Evidence 


06 


(2.8%) 


Validating 


22 


(11.4%) 


Total 


193 


(100%) 



Changes in Reasoning Over Time 
Being both a dynamic and changing process, one would 
expect to find different patterns of behavior at different 
points during independent reading. In this analysis, the 
sequential coding of each comment in the protocols, permitted 
a closer look at how these processes change in the course of 
a single reading. Each protocol was divided into three 
segments : comments before the task began, comments during the 
reading activity, and comments after the last word was read. 
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BEFORE: Three of the five subjects who participated in the 
think aloud procedure in this study, began their reading 
tasks simply plunging in, without any preparatory 
hypothesizing about the passage, i.e., without any reasoning 
or mental operations. The other two subjects were more 
reflective, even before the reading task. Before reading, all 
four (3.7%) comments reflected hypothesizing about what the 
text would be about. No other reasoning operations were 
evident in the protocols. 

DURING: Considerable consistency in the reasoning operations 
that occur from one segment to the next, with direct use of 
schemata and hypothesizing dominating. Questioning operations 
reflecting uncertainties and incomplete ideas that the reader 
has at any point in reading were also used more during the 
early stages of reading and became scarce toward the end. 
AFTER: After they had completed reading a text, all subjects 
engaged in the mental activity of validating or reaffirming 
the accuracy of information in the text (75%) . In addition, 
two subjects made comments reflecting their focus on their 
own experience (using schemata personalized) . 



Monitoring Behaviors in Reading 
The process of reading involves a variety of self- 
regulatory mechanisms which monitor meaning construction. 
Langer (1986:83) argues that "monitoring behaviors serve as a 
"third eye" to see that meanings make sense and to flag 
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problems when messages become confused" . Comprehension 
monitoring, employed by the readers in the process, serve a 
dual purpose. Firstly, when necessary, they help readers 
become aware of the cognitive activities involved in the 
process, and secondly, such mental activities help in the 
development, refinement, and organization of readers' ideas. 
It is thus safe to argue that the ability to monitor one's 
comprehension, the ability to detect comprehension 
breakdowns, and consequently utilize appropriate fix-up 
strategies, are important aspects of every reading 
experience. 

Monitoring strategies were examined in termfj of 
different concernsi of the reader: These categories, with 
examples of thei?r use, are illustrated in ?igure 6. 
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Figure 6 

Monitoring Behaviors in Reading 



I~ Awareness of Own approaches: 

a) aware of task goals / demands . "I ' 11 read it again," "So, 
let's just review," It's the vocabulary that 1 basically 
need, I got the conjugation down." 

b) Aware of subgoals: I don't know how to get to this word's 
meaning. " 

c) Aware of genre/discourse: "that's someone talking about 
the Nile, " "now we go to the letter which is written in a 
month which I don't know," "peace be with you, that's a 
greeting. Let's see, this is a letter." 

d) Aware of grammatical routines and mechanical features: 
"this must be the past tense of something," "So, that's 
whatever that called, tense, ah, it is called "passive," "I 
don't know what to do with this sentence." 

e) Aware of available or unavailable lexical repertoire: "I 
didn't know what that word means," "it's a term like "while" 
or something," "I don't understand this, anyway." 

f) aware of statements of meaning: I don't know what to do 
with this paragraph. " 

g) aware of refinements of meaning: "It ' s the capital of 
Egypt, I know that," "Ah, got that one," "ya, it's the Middle 
East, Jerusalem is in the Middle East." 
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II- Uses self-regulating mechanisms 

a) formulates task/topic goals: "I'll try to translate this." 

b) formulates subgoals : "ok, I'm confused. Let's go to some 
background information," "I don't know some words here, I'll 
look up the vocabulary. " 

c) uses genre/discourse features: No examples. 

d) uses grammatical routines or mechanical features: No 
examples . 

e) makes lexical choices: No examples. 

f) states meaning: "he wants to visit Jerusalem," ""so he's 
decided to take the trip." 

g) refines meaning: "oh, hold on, may be there are two dams," 
"I don't get it quite well, I"ll keep reading." 
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GENERAL PATTERNS 
By the nature of the protocol data gathered, all of the 
comments recorded 7_'*epresent some degree of self -awareness or 
monitoring of what one is doing while reading. Generally 
speaking, foreign language readers' comments reflecting 
av/areness of one or another idea 07: strategy exceeded those 
reflecting use by about sixfold (85% to 15%) . In the former, 
the reader is more consciously aware of the choices 
available to choose among these alternatives in a strategic 
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way. In the latter, there is monitoring of what is being 
done . 

Strategies Used in Reading 
In the process of meaning development, foreign language 
readers, like all other readers, use a a variety of broad 
strategies. Strategies are deliberate, cognitive actions that 
can be accessed for a conscious report (Paris, Lipson, & 
Wixson, 1983). Most researchers argue that the strategies and 
the phases or stages of their use, are interrelated and 
recursive rather than representing a linear sequence (Flower 
& Hayes, 1980; Spiro, 1980). Four broad strategies were 
identified in the protocols of the present study (Figure 7) 
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Figure 7 
Strategies Used in Reading 



1 (G) generating ideas~~becoming aware of relevant ideas and 
experience. 

2 (RF) Formulating and refining meaning- -drawing on personal 
experience, linking concepts or propositions, and 
paraphrasing, and summarizing. 

3 (EM) Evaluating meaning — reacting, and monitoring the 
development of meaning. 

4 (RV) Revising — knowing meaning has broken down, and taking 
appropriate measures . 
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Remarks focusing on formulating meaning occurred most 
frequently (63%), followed by revising (22%), evaluating 
(10%) , and generating (5%) . 

Knowledge Sources 
Following Lange-»- s (1986) system of meaning 
construction, three knowledge sources were identified 
(Figure 8) : knowledge of genre, knowledge of the content or 
ideas themselves, and knowledge drawn from linguistic 
material of the text. 
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Figure 8 
Knowledge Sources 



(G) genre--ref erence to the specific genre and the 

organizational structure and presentation of ideas 

peculiar to that genre (5%) . 
(C) content--reference to the topic itself (75%) . 
(T) text — reference to the linguistic material contained in 

the text — syntax, vocabulary, cohesive ties from the text 

and reworded it (20%) . 



These sources simultaneously impact the reader's 
dev. loping understanding. Genre expectations about both 
content and text, which are capable of shaping the reader's 
perception from the beginning of the reading, accounted for 
the least number of communication units or coxranents (5%) . The 
greatest proportion of comments fell into the content 
category (75%), with the remaining (20%) reflecting concern 
with text. 

Summary 

The data presented in this analysis, supplemented by the 
qualitative analysis of readers' recall protocols, provide 
our most direct look at the processes underlying foreign 
language learner's reading performance. 
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In their reading, the primary concern of subjects was 
with the meanings they were developing. Across the various 
dimensions of the analysis, this emerged in their focus on 
hypothesizing, using schemata, formulating and refining 
ideas, focusing on specific content, and validating of the 
text-model or the text-v/orld that was being developed. 

In general, patterns of behavior were somewhat different 
at different points in the reading process — with more 
questioning and hypothesizing at the beginning, and more 
using of what they knew or were learning being reported at 
later points in time. 

The general approaches of behaviors during meaning 
construction were were predominantly focused on awareness 
rather than use. In general, readers' comments focused on 
thoughts about awareness — about the options they might have 
available, rather than on what they were actually doing, 
beginning foreign language readers were more likely to 
reflect forward or back on the ideas in the text as well as 
their own ideas than they were to simply provide running 
comments and remarks . 

Though the particular emphasis varied from text to text, 
the strategies taken by individual readers in constructing 
meaning (their developing text-worlds or text models) were 
consistent across texts. Most comments reflected concern with 
formulating and refining meaning rather than generating 
ideas. Generating ideas, by definition, is a limited category 
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due to the nature of the reading task. The reader, unlike the 
writer, does not have to provide new material but rather 
focuses more on adapting the developing understandings to fit 
the author's message. 

As they construct meanings, foreign language readers 
rely on a variety of strategies and approaches, most of which 
seem to be useful and used across a variety of specific 
texts . 

The analysis of beginning foreign language readers* 
behaviors while reading leads to the conclusions that the 
behaviors are varied and complex and that they change 
slightly with text and topic. 

The data reported in this section provide strong 
evidence that in reconscructing text meaning, even beginning 
foreign language readers employ reasoning and mental 
operations, monitor their comprehension, and draw upon 
different sources of knowledge. The reasoning operations and 
the strategies are of particular interest in that they may be 
the focus of instructional interventions. Readers may be 
trained in the different aspects of mental and reasoning 
operations as well as other prereading activities that may 
foster comprehension. 
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